8.7. Capstone
In one glance
- You will: Turn the reference agent into your own domain agent, one boundary at a time, then prove it reproduces from a clean clone — by a reviewer, or by you on a machine that knows nothing.
- You need: Part I completed for the developer capstone; Chapters 5-7 additionally completed for the platform capstone.
- Time: about 60 minutes to plan and measure it; the build itself is open-ended, hands-on.
Choose a developer capstone or a platform capstone. The developer capstone ends with tested Python behavior, evaluation evidence, and the A2A handoff; Kubernetes is not required to complete Part I. The platform capstone adds the deployment, observability, and recovery criteria below. Use the completed reference as a platform starting checkpoint if that is your learning goal.
What is the capstone goal?
A capstone is the closing project you build yourself: one agent, end to end, with this course as the reference.
Create a domain-specific agent platform that preserves the course's open contracts while replacing the fictional AgentOps Agent domain with a problem you understand. Examples include a developer-support agent, data-quality investigator, security triage assistant, or internal platform runbook copilot.
One worked pivot makes the boundaries concrete. To turn the reference into a developer-support agent, thread a single capability through the layout the repository already has:
- Seed
agents/data/sql/withticketrows instead of theINC-*incidents. - Add a typed
get_ticketread tool intools.pymirroringget_incident: validate the id, cite provenance. - Re-list
get_ticketin the MCP allowlist so the gateway will pass it. - Add one case to
agents/python/evals/ops.evalset.jsonthat exercises the new trajectory.
That is one read capability crossing data, tools, gateway policy, and evidence — the same four boundaries every milestone touches.
Scope the work before you start, because the ten milestones below do not all need the same machinery:
- Milestones 1-6 stay on your laptop: configured model access and the repository gate, nothing else.
- Milestone 7 needs the host agentgateway of Chapter 5.
- Milestone 8 needs the shared k3d cluster of Chapter 6.
- Milestone 9 needs the telemetry stack of Chapter 7.
Keep the application and platform software open source. Label Gemini as a proprietary hosted model service, document quota use, and retain the optional Ollama/open-weight path and offline tests.
This is an extension project, not a file-by-file reconstruction exercise. Start from the completed reference on main, create a focused branch, and change one boundary at a time while the existing gates remain green.
What is sufficient for the developer capstone?
Submit a small domain agent with evidence another Python developer can reproduce.
mise run lab -- check 8
| Criterion | Evidence |
|---|---|
| Useful bounded task | A domain question, its expected result, and an explicit non-goal |
| Typed, grounded tools | Read-only seed data, valid and invalid input checks, and evidence provenance |
| State and action authority | Conversation isolation; denied, approved, and replayed simulated actions |
| Bounded execution | A workflow whose termination and tool permissions you can explain |
| Evaluation | Labeled cases, missing/invented evidence failures, and results tied to source and model configuration |
| Untrusted evidence | Fenced retrieval, labeled injection cases, one documented miss, and the control that contains it |
| Interoperability | A read-only MCP tool listing and an A2A discovery/request demonstration using the reference handoff |
Each criterion must include the command and observed outcome, or an explicit evidence gap. Offline calibration does not substitute for a live-model quality result. Kubernetes, gateway installation, dashboards, and cloud deployment are not required for this completion point.
The remaining sections describe the larger reference-domain port and platform capstone. Platform learners may start directly from the completed reference after 4.8. Developer Handoff.
What is your starting state?
Before changing the domain, establish a reproducible learner baseline and a clean working tree.
mise run install
mise run doctor
mise run format:core
mise run check:core
mise run test
git status --short
A passing baseline is: every task above exits without error, mise run test reports its coverage summary, and git status --short prints nothing.
Record the commit you started from and the passing test/coverage summary. Keep the immutable seed/writable state split: committed input is reproducible, runtime state is disposable, and approved writes never dirty the seed.
mise run install installs the contributor environment for extending the completed reference; do not precede it with bare mise install, which would fetch every optional tool. Later milestones each add a scoped install or doctor:
cd agents/python && mise run config:checkfor model-backed work in milestones 1-6.cd agents/python && mise run install:evalfor the full evaluation evidence in milestone 6.mise run doctor:gatewayfor milestone 7.mise run install:platformandmise run doctor:platformfor milestone 8.mise run install:maintainerbefore the final all-surfaces gate.
Run the reference once with your selected provider before changing behavior; this consumes model quota on Gemini:
(cd agents/python && mise run config:check)
(cd agents/python && mise run web)
Open http://127.0.0.1:8002, ask one read-only question, and save the provider, model, prompt, Events trajectory, and result as your baseline evidence. Stop the server with Ctrl+C.
Which design decisions must you write down?
Keep the design brief to one page or one pull-request description, but make these decisions explicit:
| Decision | Question your design must answer |
|---|---|
| User and outcome | Who uses the agent, for which bounded decision, and what should remain deterministic software? |
| Domain data | Which inputs are immutable seed, which state is writable, and which data must never enter model context? |
| Authority | Which operations are reads, which are guarded writes, and who can approve each write? |
| Model path | Which provider and model meet your quality, memory, and cost limits, and what evaluation supports that choice? |
| Tool boundary | Which capabilities stay in-process, which cross MCP, and which must never be delegated? |
| Agent boundary | Is one agent enough, or does an A2A/delegation boundary have a measurable ownership benefit? |
| Failure policy | What are the deadlines, retries, budgets, safe errors, and no-fallback cases? |
| Evidence | Which test, evaluation, trace, metric, and audit record proves the platform works? |
Prefer the smallest platform that satisfies those answers. Adding another framework, database, broker, vector store, or cloud service is not a capstone achievement unless it removes a demonstrated constraint.
Which files will you normally change?
Use the existing flat ownership boundaries:
| Boundary | Typical files |
|---|---|
| Immutable domain input | agents/data/incidents.db, agents/data/sql/, agents/data/logs/, agents/data/runbooks/, agents/data/skills/ |
| Trusted types and state | agents/python/src/agent/models.py, data.py, longterm.py |
| Read and knowledge tools | tools.py, skills.py, memory.py, retrieval.py, mcp_server.py |
| Guarded writes | actions.py, guardrails.py, database schema/triggers |
| Agent behavior | composition.py, model.py, workflow.py, delegation.py, report.py |
| Quality evidence | agents/python/tests/, agents/python/evals/, load/ |
| Data plane | infra/agentgateway/{host,k3d,gke}/, infra/kagent/ |
| Platform and telemetry | infra/k8s/, infra/observability/, infra/mlflow/ |
| Learner/operator contract | README.md, AGENTS.md, relevant docs/ pages, component READMEs |
The developer-support pivot from the first section lands in four of those rows: immutable domain input, read tools, data plane, and quality evidence.
Do not rename every file merely to make the project look new. Rename domain language where it improves clarity; preserve stable protocol and task contracts where they already fit.
What is the dataset contract you must replace?
agents/data/ is a payload with its own build, its own gate, and no page of its own until this one. Milestone 1 replaces it wholesale, so read its contract before you edit a row.
Four things define it:
sql/schema.sql— the tables the agent reads and writes:services,incidents, and the append-onlyaudit_log.sql/seed.sql— deterministic rows with fixed ids and timestamps. Fixed values are what make two evaluation runs comparable.incidents.db— generated from those two files and committed, so a fresh clone answers questions offline.runbooks/*.md,logs/*.log, andskills/*/SKILL.md— the files those rows point at.
The database is a build artifact you must not hand-edit. From agents/data:
mise run build # rm -f incidents.db, then replay schema.sql and seed.sql into it
mise run check # rebuild into a temporary file, compare, then check four referential rules
check is not a formality. It enforces the four rules a hand-edited dataset breaks silently:
- The committed
incidents.dbis byte-for-byte what the SQL produces — with the writer version pinned inagents/data/mise.toml, because anothersqlite3build writes different bytes. - Every
incidents.servicevalue exists inservices. - Every
incidents.runbookvalue has a matchingrunbooks/<slug>.md. - Every
logs/<name>.lognames a service that exists.
The root task mise run check:data runs exactly that, and mise run check:core depends on it — so the pre-commit hook already refuses a seed whose database, rows, runbooks, and logs disagree. Your replacement domain inherits that gate the moment your rows land in the same shape.
Two invariants outlive any domain: identifiers are validated against a pattern in models.py before anything reads them, and the committed seed is never mutated at runtime — the agent copies it into AGENT_STATE_DIR first.
How do you port the test suite?
Milestone 1 is not a commit-sized change, and its entry gate deserves a current inventory. Measure it in your own clone before you plan anything:
cd agents/python
rg -l -e 'INC-[0-9]' -e '\b(checkout|payments|inventory|api-gateway|search)\b' tests/test_*.py
Do not turn that output into a target of zero. A literal can be the behavior under test—an MCP example, policy target, expected trajectory, or redaction case—rather than duplicated setup. The repository's portability audit counts those explicit examples per file and rejects growth beyond its ratcheted allowance.
tests/conftest.pyinstalls an autouse fixture that copies the whole dataset directory for every test, so every test runs against your seed whether it asserts on the domain or not.- The floor does not move while you work:
mise run testkeeps--cov-fail-under=95on combined line-and-branch coverage. Deleting the tests that no longer apply is not a strategy — the floor notices.
Port in this order, running mise run test after each step rather than once at the end:
- The domain-owned content. Rewrite
sql/schema.sql,sql/seed.sql, runbooks, logs, and runtime skills, then run the data build and gate. Everything below reads this content, so a seed that changes twice costs you the work twice. - The typed vocabulary.
src/agent/domain.pyownsDomainVocabularywithincidents,services,runbooks, anddependency_edges;REFERENCE_DOMAINowns the reference names. Test support intests/domain.pyownsPIVOT_DOMAINandadapt_dataset, proving a copied seed and its files can cross that one adapter surface. Replace the production vocabulary for generic setup while keeping behavior-specific literals visible where they are asserted. - The validators.
models.pypins^INC-\d+$for incident ids and a lowercase slug pattern for services and runbooks. If your identifiers areTCK-1024or a UUID, change the pattern here before touching a tool: every read tool, the structured report, and the MCP schema inherit it. - The boundary evidence.
test_domain_portability.pyexercises the reference and pivot vocabulary across reads, guarded writes, protocol, policy, and evaluation validation. The eval-set JSON files still carry domain prompts and expected trajectories; change them only after the seed and vocabulary are stable.
The adapter is intentionally test support, not a project generator or a promise that every domain has the same schema. It exposes the coupling the reference already knows about and gives the ratchet one owner. Your production types, tools, seed, and evidence still need deliberate domain design.
Your turn: how do you prove the domain coupling before milestone 1?
Required drill — the Chapter 8 checkpoint asks for its result. Break one seed row and let the suite show you the blast radius, instead of planning a port around a guess.
- Mode:
temporary experiment. - Goal: change exactly one identifier in the seed, rebuild the database, and read which tests fail and why. You are measuring coupling here, not fixing anything.
- Files to touch:
agents/data/sql/seed.sqland the generatedagents/data/incidents.db— rename one incident id, for exampleINC-002toINC-902— thencd agents/data && mise run build, because the tests read the generated database and not the SQL. - Preflight: require
git diff --quiet -- agents/data/sql/seed.sql agents/data/incidents.db; both are experiment targets because the build regenerates the database. - Gate that proves completion:
cd agents/python && uv run pytest -qnow fails, and the failures name the identifier you changed rather than an unrelated stack. - Final state: from the repository root, run
git restore -- agents/data/sql/seed.sql agents/data/incidents.db, then confirmcd agents/python && uv run pytest -qand rootmise run check:dataare green. The focusedgit diff --quiet --preflight must pass again.
Which milestones should you deliver?
Ten milestones, in order. Each has one signal that tells you it is finished; milestone 1 is a multi-commit port, and the nine after it are commit-sized.
- Bound the domain. The largest milestone by far: replace domain-owned content, update the typed vocabulary and validators, then review every ratcheted literal the portability audit reports. Done when your sanitized seed passes
mise run check:data, the reference/pivot portability checks pass for your vocabulary, no unexplained domain term exceeds the ratcheted allowance, andmise run testis green at the unchanged 95% combined line-and-branch coverage floor. Document one useful user outcome plus one case where deterministic code is better than an agent. - Implement evidence-backed reads. Done when one typed read tool returns a record with its provenance — the source the model can cite — and an invalid identifier is refused by a test. Add at least one typed read tool and one runbook/skill or retrieval path, and validate identifiers before filesystem/database access.
- Implement one guarded write. Done when a write without confirmation fails a test, and an
UPDATEorDELETEagainst the audit table is rejected. Require explicit confirmation and rationale, validate the target, and commit state plus audit evidence in one transaction. - Compose the agent. Done when
mise run webanswers your own domain question with your configured provider and its Events timeline shows the tools you registered. Update the instruction, structured report, tool registry, budgets, retries, and safe errors. Keep the selected provider explicit and imports free of network/destructive work. - Expose open boundaries. Done when your new read is listed by your MCP server and its user-facing capability is described by an A2A skill. Publish the appropriate reads through MCP, and keep writes in-process unless you can preserve confirmation authority.
- Prove behavior. Done when
mise run testand your model-backed evaluation cases both pass, including at least one adversarial case. Those cases check tool trajectories and grounding rather than prose style alone. - Govern the host path. Done when
mise run smoke:hostproves the MCP, A2A, model, and metrics routes against your own policies. Update the agentgateway MCP allowlist/policies and run the loopback wrapper. - Deliver locally on Kubernetes. Done when your workload is Ready on the shared k3d cluster and no public service exists. Update the image/manifests, deploy with kagent, and verify private ClusterIP/port-forward access, persistence, probes, quotas, and NetworkPolicies.
- Close the operations loop. Done when one request leaves a trace, a useful metric/dashboard view, an evaluation result, and an approved-action audit row you can point at. Write the scoped cleanup procedure alongside them.
- Make it teachable. Done when another learner can reproduce the baseline, run one successful path, diagnose a failure, and clean up without private knowledge. Update the human and agent documentation until that is true.
The milestones follow the same lifecycle arc as the course, and each one returns to the repository gate before the next begins:
flowchart TD
M1["1 Bound the domain<br/>Ch. 2-3"] --> M2["2 Evidence-backed reads<br/>Ch. 3"]
M2 --> M3["3 One guarded write<br/>Ch. 4.5"]
M3 --> M4["4 Compose the agent<br/>Ch. 2-3"]
M4 --> M5["5 Expose MCP / A2A<br/>Ch. 3"]
M5 --> M6["6 Prove behavior<br/>Ch. 4"]
M6 --> M7["7 Govern host path<br/>Ch. 5"]
M7 --> M8["8 Deliver on k3d<br/>Ch. 6"]
M8 --> M9["9 Close the ops loop<br/>Ch. 7"]
M9 --> M10["10 Make it teachable<br/>Ch. 8"]
Gate{"core gates green?<br/>format:core · check:core · test"}
M1 -.-> Gate
M10 -.-> Gate
Gate -.next milestone.-> M1
Each milestone is taught by the chapter labelled on its node, so a stalled milestone tells you exactly where to re-read: milestone 3 in 4.5. Guardrails for the guarded write, milestone 7 in Chapter 5 for the host path, milestone 8 in Chapter 6 for local delivery, and milestone 9 in Chapter 7 for the operations loop. Keep the core gate green after every milestone, then add the relevant runtime proof. Run the full maintainer gate before the final handoff.
Which contracts should remain stable?
Four things stay put unless you have a reason: the explicit model-provider contract, the port map, the seed/state split, and confirmation plus audit on every write.
Deeper: the eight contracts, in full
AGENT_MODEL_PROVIDER=openai-compatiblewith direct Ollama athttp://127.0.0.1:11434/v1remains the optional local alternative.- Chapter 5 changes
OPENAI_BASE_URLtohttp://127.0.0.1:4000/v1; provider selection does not double as deployment topology. - MCP reads use
:3000, A2A uses:3001, the model uses:4000, gateway metrics use:15020, and host readiness uses:15021. - Immutable seed and writable runtime state remain separate.
- State-changing tools require confirmation and append audit evidence atomically.
- Telemetry content capture remains
falseby default. - Host listeners are loopback-published through the digest-pinned wrapper; native Linux uses a bridge-scoped relay for loopback backends, Compose scrapes gateway metrics directly on that scoped bridge, and Kubernetes services remain private.
- Offline tests never require a model, provider account, cluster, or cloud resource.
You may change a stable contract only when the capstone's outcome requires it. State the incompatibility, migration, updated tests, and learner impact explicitly.
Which gates prove each layer?
Every layer you claim has one minimum gate. Run the row before you write the claim down.
| Layer | Minimum gate |
|---|---|
| Learner repository | mise run format:core, mise run check:core, mise run test |
| Final repository | mise run install:maintainer, then mise run format, mise run check, mise run test, and mise run scan |
| Model behavior | Configuration check plus one recorded read-only trajectory, with provider and model named |
| Host data plane | mise run doctor:gateway, the three host processes, then mise run smoke:host |
| Agent behavior | Deterministic tests plus recorded model-backed eval cases against an explicit model |
| Kubernetes | mise run doctor:platform, both overlay renders, local deployment, readiness, private-service, and persistence checks |
| Observability | One correlated trace, gateway metric, evaluation run, and audit record |
| Optional GCP | mise run doctor:gcp and a reviewed tofu plan; apply/destroy only with explicit approval |
A screenshot is supporting evidence, not a gate. Prefer commands, machine-readable output, tests, traces, and artifact digests that another person can reproduce.
What evidence should your final submission contain?
One concise handoff document: what you built, the commits, and the artifact for every layer you claimed above.
Deeper: the eleven items the handoff lists
Provide one concise handoff with:
- The user problem, non-goals, and architecture/authority decisions.
- The starting and final Git commit identifiers.
- The selected provider/model/base URL and the installed model digest when using Ollama.
- The deterministic test summary and branch-coverage result.
- The evaluation cases, exact tool-trajectory score, and known model failures.
- A sanitized trace identifier or export showing model, tool, and gateway spans.
- A gateway smoke result and the policies/allowlist added for your tools.
- Kubernetes resources, image digest, readiness result, and proof no public service was created.
- One approved write with its confirmation, state change, and audit row.
- Security-scan results, residual risks, and deliberately absent production controls.
- Cleanup commands and confirmation that disposable processes/resources were stopped.
Do not submit credentials, raw sensitive prompts, private logs, .env, runtime databases, or generated key material.
For an optional portfolio handoff, keep the document to six short sections:
- Architecture: one diagram plus the user, trust, data, and approval boundaries in words.
- Exact commands: the clean-clone install, gate, demo, and cleanup commands you actually ran.
- Evaluation lineage: source revision, dataset/eval-set digests, provider and model identity, and the recorded result.
- Trace and audit evidence: sanitized identifiers that connect one request to its tool spans and approved write.
- Limitations: unsupported platforms, absent production controls, known model failures, and residual risks.
- Short demo: one bounded success and one diagnosed failure; link reproducible evidence rather than only a video.
How do you make the offline evidence independently verifiable?
The repository can create a small local manifest after your tracked changes are committed and the working tree is clean:
mise run course:evidence
The command reruns check:core and test, then writes .agents/tmp/course-completion.json. It records the exact Git revision, the fixed gate inventory, and SHA-256 digests of the locks, seed database, and evaluation set. It deliberately records no command output, environment values, prompts, runtime databases, credentials, or telemetry.
Give that JSON file to a reviewer who has checked out its exact revision. From that clean clone, the reviewer runs:
mise run course:evidence:verify -- /path/to/course-completion.json
Verification rejects a different revision, tracked or untracked source changes, changed artifacts, or a changed gate inventory, then reruns both deterministic gates. This is reproducible local evidence, not an identity proof, certificate, hosted grader, or claim about model, gateway, Kubernetes, observability, or cloud results.
How is the capstone assessed?
Nobody grades this but you, or a reviewer you choose. Use the rubric below as a self-review checklist and read its points as weights.
Score the evidence, not the number of tools. A smaller platform with clear authority and reproducible proof should outperform a larger stack with hidden assumptions.
Deeper: the hundred-point self-review rubric
| Area | Points | Full-credit evidence |
|---|---|---|
| Problem and architecture | 10 | Bounded outcome, explicit non-goals, simple ownership boundaries |
| Open-source boundary | 10 | OSS software, explicit hosted-service costs, and offline/local alternatives |
| Data and tools | 15 | Trusted types, provenance, least privilege, seed/state separation |
| Authority and security | 15 | Guarded atomic write, audit proof, PII/injection defenses, secret hygiene |
| Quality | 15 | Deterministic tests, at least 95% combined line-and-branch coverage, adversarial regressions |
| Evaluation | 10 | Recorded trajectories, grounding criteria, explicit model/version evidence |
| Gateway and interoperability | 10 | MCP/A2A/model routes, fail-closed policy, loopback smoke proof |
| Platform and operations | 10 | Reproducible local k3d delivery, persistence, private networking, telemetry |
| Documentation and cleanup | 5 | Reproducible quickstart, diagnosis, limitations, and safe teardown |
How do you clean up safely?
Stop host processes and the detached gateway:
mise run gateway:host:stop
mise run observability:down
(cd agents/python && mise run data:reset)
For Kubernetes, this deletion also removes the course PVCs and their state. Review retained evidence and backups before using your deployed profile (local-gemini below, or local for Ollama):
test "$(kubectl config current-context)" = k3d-local
(
cd infra
skaffold delete --filename skaffold.yaml --profile local-gemini
)
Do not delete a shared cluster, PVC, cloud resource, or generated evidence until you have reviewed its ownership and retention requirement. An optional GCP plan creates nothing; any approved apply needs a separately reviewed destroy and cost check.
What proves this page worked?
The strongest proof is another person. Ask them to follow your documented path from a clean authorized clone: pass the base gate, run one grounded request with your selected provider, exercise the governed host route, inspect your evidence, and clean up — without asking you for an undocumented credential or step.
This course is self-paced and openly licensed, so it owes you a version you can finish alone. The solo handoff tests the same claim — that your instructions, not your machine, carry the knowledge:
- Clone your repository into a directory you have never worked in. A clean
HOME, a fresh user, or a container is stronger, because it also drops your shell configuration and your tool caches. - Carry nothing across: no
.env, no.state/, no hand-copied caches, no shell history. Whatever your documentation does not name, you do not have. - Type only commands your own README and pages contain. The moment a step fails or needs something unwritten, stop, fix the documentation, and restart the run from the top.
- Finish the reviewer's path: gate, one grounded request, the governed host route, the evidence, cleanup.
The one failure a solo run cannot catch is the one you can name in advance: you already know what your own words mean. So when a reviewer is available, use them — that is the stronger evidence. One of the two runs is the terminal gate, and skipping both is not an option.
Your capstone is complete when that handoff succeeds and the final diff still passes:
mise run install:maintainer
mise run format
mise run check
mise run test
mise run scan
git status --short
You are done when:
- The five
misetasks above pass andgit status --shortprints nothing. - A matching clean clone accepts the local completion manifest with
mise run course:evidence:verify. - Every milestone you claim has its done-signal met, not just a description written.
- One request of your own left a trace, an evaluation result, and an audit row you can show.
- The coupling drill ran and left nothing behind: the suite went red on your one seed edit and green again after restoring both the SQL and generated database.
mise run check:datapasses against your own seed, and every remaining original-domain literal has a documented reason in the portability audit.- Either a reviewer or a from-scratch clone with nothing carried over finished the whole path — baseline, grounded request, host route, cleanup — without an undocumented question or an unwritten step.
Continue to the optional OSS maintenance tour when you plan to publish or sustain the result, or return to the course home.